2001a), “Evaluating forecasting methods

I examined six ways of selecting forecasting methods: Convenience, “what’s easy, ” is inexpensive, but risky. Market popularity, “what others do, ” sounds appealing but is unlikely to be of value because popularity and success may not be related and because it overlooks some methods. Structured judgment, “what experts advise, ” which is to rate methods against prespecified criteria, is promising. Statistical criteria, “what should work, ” are widely used and valuable, but risky if applied narrowly. Relative track records, “what has worked in this situation, ” are expensive because they depend on conducting evaluation studies. Guidelines from prior research, “what works in this type of situation, ” relies on published research and offers a low-cost, effective approach to selection. Using a systematic review of prior research, I developed a flow chart to guide forecasters in selecting among ten forecasting methods. Some key findings: Given enough data, quantitative methods are more accurate than judgmental methods. When large changes are expected, causal methods are more accurate than naive methods. Simple methods are preferable to complex methods; they are easier to understand, less expensive, and seldom less accurate. To select a judgmental method, determine whether there are large changes, frequent forecasts, conflicts among decision makers, and policy considerations. To select a quantitative method, consider the level of knowledge about relationships, the amount of change involved, the type of data, the need for policy analysis, and the extent of domain knowledge. When selection is difficult, combine forecasts from different methods